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Onbemande  vliegtuigen  (Unmanned  Aerial  Vehicles,  UAV’s)  zijn  al  enkele  tientallen  jaren  in 
gebruik  bij  diverse  strijdmachten  in  de  wereld.  De  laatste  tijd  neemt  hun  belang  aanzienlijk  toe. 
Zowel  in  de  Golfoorlog  als  in  het  voormalige  Joegoslavie  zijn  ze  in  ruime  mate  ingezet.  Er  bestaat 
inmiddels  een  groot  scala  aan  types,  van  zeer  grote,  hoogvliegende  radarverkenners  tot  kleine 
laagvliegende  vliegtuigjes,  helikoptertjes  en  visueel  bestuurde  raketten.  Hoewel  het  vliegen  vaak 
geautomatiseerd  is,  zijn  er  situaties  waarin  een  menselijke  operator  de  besturing  overneemt.  In  het 
geval  dat  er  een  videocamera  aan  boord  is  en  deze  sensor  van  de  UAV  handmatig  bestuurbaar  is, 
kan  er  onduidelijkheid  ontstaan  over  de  relatie  tussen  vlieg-  en  kijkrichting.  Een  van  de  mogelijke 
oplossingen  is  de  richting  van  de  UAV-sensoren  te  koppelen  aan  de  hoofdbewegmgen  van  de 
operator,  waardoor  een  duidelijker  referentiekader  ontstaat.  De  sensorbeelden  worden  hierbij  recht 
voor  de  ogen  van  de  operator  op  een  Head  Mounted  Display  (HMD)  afgebeeld.  Alvorens  te  testen 
of  deze  methode  voordelen  biedt  boven  handmatige  besturing  is  het  zaak  te  bestuderen  welke 
factoren  de  stuurprestatie  beinvloeden  bij  een  dergelijke  ondersteuning  van  de  stuurtaak,  zodat 
hoofdgekoppelde  camerasturing  optimaal  afgezet  kan  worden  tegen  handbesturing.  Een  aantal  van 
deze  factoren  is  in  dit  onderzoek  onderzocht.  Deze  factoren  zijn  verder  ook  van  belang  voor 
bestuurders  die  wel  in  hun  voertuig  zitten  en  ondersteund  door  visuele  hulpmiddelen  dit  voertuig 
besturen  (gemedieerd  zicht).  De  Apache  AH-64  helikopter  beschikt  bijvoorbeeld  over  een  op  de 
neus  bevestigde  en  hoofdgestuurde  infra-rood  sensor  en  helderheidsversterker.  De  metingen  leveren 
verder  informatie  die  voor  simulatoren  in  het  algemeen  van  belang  is. 

Het  onderzoek  werd  in  de  TNO-TM  vliegsimulator  uitgevoerd,  waar  een  gesimuleerde  UAV-vlucht 
door  een  bochtig  en  in  hoogte  varierend  geuldal  werd  uitgevoerd.  De  taak  van  de  proefpersonen 
was  een  slalomparcours  gemarkeerd  door  gekleurde  bomen  zo  goed  mogelijk  af  te  leggen,  waarbij 
een  vaste  afstand  van  de  bomen  en  een  vaste  hoogte  boven  de  grond  moest  worden  aangehouden. 

In  het  experiment  werd  gekeken  naar  de  factoren  beeldhoek,  camera-vertraging,  monoculaire  vs. 
stereoscopische  presentatie  en  HMD-type.  Om  de  beeldhoek  te  kunnen  varieren  werd  een  HMD 
nagebootst  door  een  in  grootte  instelbaar  (gesimuleerd)  venster  kijkrichting-gestuurd  over  de 
geprojecteerde  simulatorbeelden  te  laten  bewegen.  Een  van  de  beeldgroottes  die  op  deze  manier 
werd  aangeboden  correspondeerde  met  de  grootte  van  de  eveneens  in  dit  experiment  gebruikte 
Virtual  IO  i-glasses  HMD  waardoor  een  vergelijking  mogelijk  werd. 

De  resultaten  laten  zien  dat  het  gebruik  van  de  gesimuleerde  HMD  een  significant  betere  stuurpres¬ 
tatie  oplevert  dan  gebruik  van  de  echte  HMD;  dit  kon  echter  niet  toegerekend  worden  aan 
verschillen  in  resolutie.  De  prestatie  bij  een  beeldhoek  van  17°  is  significant  lager  dan  bij  34  of  57  . 
Een  cameravertraging  van  50  ms,  typisch  voor  de  mechanische  vertraging  die  bij  servosystemen 
optreedt,  had  een  significante  invloed  op  de  stuurnauwkeurigheid.  En  als  laatste:  de  monoculaire 
and  stereoscopische  presentatiewijzen  leidden  niet  tot  verschillende  prestaties. 
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SUMMARY 

Military  use  of  Unmanned  Aerial  Vehicles  (UAVs)  is  gaining  importance.  Video  cameras  in 
these  devices  are  often  operated  with  joysticks  and  their  image  is  displayed  on  a  CRT .  In  this 
experiment,  the  simulated  camera  of  a  simulated  UAV  was  slaved  to  the  operator  s  head 
movements  and  displayed  using  a  Helmet  Mounted  Display  (HMD).  The  task  involved 
manoeuvring  a  UAV  along  a  winding  course  marked  by  trees.  The  influence  of  several 
parameters  of  the  set-up  (HMD  optics,  Field  of  View  (FOV),  image  lag,  monocular  vs. 
stereoscopic  presentation)  on  a  set  of  flight  handling  characteristics  was  assessed.  To  enable 
variation  of  FOV  and  to  study  the  effect  of  the  HMD  optics,  a  simulated  HMD  image 
consisting  of  a  head  slaved  window  (with  variable  FOV),  was  projected  on  a  screen.  One  of 
the  FOVs,  generated  in  this  way,  corresponded  with  the  FOV  of  the  real  HMD,  enabling  a 
comparison.  The  results  show  that  the  simulated  HMD  yields  a  significantly  better  perform¬ 
ance  than  the  real  HMD.  Performance  with  a  FOV  of  17°  is  significantly  lower  than  with  34 
or  51°.  An  image  lag  of  50  ms,  typical  of  pan-and-tilt  servo  motor  systems,  has  a  small  but 
significant  influence  on  steering  accuracy.  Monocular  and  stereoscopic  presentation  did  not 
result  in  significant  performance  differences. 
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Afstand  bestuurd  vliegen  ondersteund  door  een  hoofd-gekoppelde  camera  en  HMD 

S.C.  de  Vries  en  P.  Padmos 

SAMENVATTING 

De  militaire  toepassing  van  onbemande  vliegtuigen  (UAV’s)  wint  aan  belangstelling. 
Beeldvormende  sensoren  in  deze  toestellen  worden  dikwijls  handmatig  bediend  door  middel 
van  joysticks  waarbij  hun  beeld  wordt  weergegeven  op  een  monitor.  Deze  wijze  van 
aansturing  zou  een  negatieve  invloed  kunnen  hebben  op  het  situationeel  bewustzijn  van  de 
operator:  de  relatie  tussen  kijkrichting  en  vliegrichting  kan  onduidelijk  worden.  In  dit 
experiment  werd  een  gesimuleerde  camera  van  een  gesimuleerde  UAV  bestuurd  door  de 
hoofdbewegingen  van  de  bestuurder,  terwijl  de  beelden  werden  getoond  door  middel  van  een 
Head  Mounted  Display  (HMD),  waardoor  een  meer  ecologische  koppeling  tussen  sturing  van 
sensor  en  waarneming  van  het  corresponderend  beeld  ontstaat. 

De  opdracht  van  de  bestuurder  was  het  zo  goed  mogelijk  manoeuvreren  van  een  UAV  langs 
een  bochtig  en  heuvelachtig  parcours  gemarkeerd  door  bomen. 

De  invloed  van  een  aantal  parameters  van  de  opstelling  (HMD  optiek,  beeldgrootte,  beeldver- 
traging,  monoculaire  of  stereoscop ische  presentatie)  op  een  verzameling  prestatie-  en 
gedragskarakteristieken  werd  bestudeerd.  Om  de  beeldgrootte  te  kunnen  varieren  en  het  effect 
van  de  HMD-optiek  te  kunnen  bestuderen  werd  een  HMD  gesimuleerd  door  over  de  in  een 
bolprojectie  weergegeven  scene  een  computergegenereerd  en  hoofdgekoppeld  venster  te 
leggen.  Een  van  de  beeldgroottes  die  op  deze  manier  werd  aangeboden  correspondeerde  met 
de  beeldgrootte  van  de  echte  HMD  die  ook  in  het  experiment  werd  gebruikt.  Hierdoor  werd 
een  vergelijking  mogelijk. 

De  resultaten  laten  zien  dat  het  gebruik  van  de  gesimuleerde  HMD  een  significant  betere 
stuurprestatie  oplevert  dan  gebruik  van  de  echte  HMD;  dit  kon  echter  niet  toegerekend 
worden  aan  verschillen  in  resolutie.  Een  hypothese  is  dat  de  in  het  eerste  geval  zichtbare 
randen  van  het  projectiescherm  en  de  overgangen  tussen  de  beeldkanalen  van  het  beeldsys- 
teem  voertuigreferenties  vormen  die  door  de  proefpersonen  zijn  gebruikt  om  hun  situationeel 
bewustzijn  te  vergroten. 

De  prestatie  bij  een  beeldhoek  van  17°  is  significant  lager  dan  bij  34  of  57  .  Een  cameraver- 
traging  van  50  ms,  typisch  voor  de  mechanische  vertraging  die  bij  servosystemen  optreedt, 
had  een  kleine  maar  significante  invloed  op  de  stuurnauwkeurigheid.  De  monoculaire  and 
stereoscopische  presentatie  leidden  niet  tot  verschillende  prestaties. 
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1  INTRODUCTION 

In  flying  some  Unmanned  Aerial  Vehicle  (UAV)  systems,  the  remote  operator’s  view  of  the 
ambient  world  is  provided  by  one  or  more  cameras  on  the  vehicle  and  one  or  more  monitors 
at  the  control  station.  In  order  to  provide  a  large  visual  field,  while  restricting  the  required 
transmission  capacity  from  vehicle  to  operator,  a  head  tracked  pan-and-tilt  camera  on  the 
UAV  combined  with  a  Head  Mounted  Display  (HMD)  is  a  potentially  attractive  solution. 
Head-tracked  control  of  the  camera’s  viewing  direction  is  more  attractive  than  joystick  control 
because,  firstly,  the  operator’s  hands  remain  free  and,  secondly,  the  operator  presumably 
knows  intuitively  and  precisely  in  what  direction  with  respect  to  himself  and  the  UAV.  Before 
comparing  head-tracked  control  and  joystick  control  it  makes  sense  to  determine  how  several 
parameters  of  the  system  affect  operator  performance,  so  that  the  comparison  can  be  made  for 
a  reasonably  optimized  set-up.  Additionally,  the  same  factors  that  affect  performance  in 
remote  control  will  also  effect  pilots  who  are  provided  with  sensor  information  in  the  same 
way  as  in  the  Apache  and  Cobra  helicopters,  where  a  HMD  is  combined  with  a  head  tracked 
pan-and-tilt  infra-red  and  image-intensifier  sensor.  The  goal  of  the  current  research  was  to 
examine  several  of  these  factors. 

The  present  report  is  aimed  at  the  question  of  the  required  instantaneous  visual  field  size  for  a 
HMD-camera  system,  for  adequately  performing  a  nap-of-the-earth  (NOE)  flight  on  a 
winding  course.  In  the  same  experiment  we  studied  the  performance  effects  of  stereoscopic 
vs.  monocular  image  presentation,  and  of  the  introduction  of  an  image  lag  due  to  delays  in  the 
camera’s  servo-motor  system. 

The  instantaneous  field  size  is  a  technically  important  parameter  in  the  design  of  a  HMD- 
camera  system,  because  it  is  related  to  parameters  such  as  spatial  resolution,  magnification, 
and  required  transmission  capacity.  Operationally  it  is  important  because  with  larger  instan¬ 
taneous  field  sizes  less  head  motion  is  required  to  anticipate  the  course  of  the  path  to  be 
flown,  and  generally  there  is  a  more  accurate  perception  of  the  vehicle’s  motion  (Padmos, 
1995;  Warren,  Blackwell,  Kurtz,  Hatsopoulos  &  Kalisk,  1991). 

In  principle,  a  stereo  image  requires  two  cameras,  which  doubles  the  required  transmission 
capacity.  However,  it  enables  better  depth  perception  than  a  image  containing  only  monocular 
information,  albeit  that  its  advantage  is  generally  small  compared  with  monocular  depth  cues 
at  distances  above  15  m  (Padmos,  1992,  1995).  One  of  the  questions  in  this  experiment  was 
whether  stereopsis  is  useful  in  low-flying  tasks  when  distances  in  this  range  are  present. 

Image  delays  due  to  camera  inertia  are  unavoidable.  In  practice  delays  can  be  restricted  to  50 
ms,  with  a  sophisticated,  powerful,  tracking  system  (Sharkey  &  Murray,  1993).  It  was 
reported  that  slaving  system  imperfections  (time  constant  0.5  s)  in  a  helicopter  simulation 
seriously  constrain  the  pilot  from  making  fast  head  movements  and  increased  the  time  to 
estimate  a  location  of  a  point  on  the  flying  path  (Grunwald,  Kohn  &  Merhav,  1991).  In  an 
object  handling  task  for  subjects  wearing  a  HMD,  Kawara,  Ohmi  and  Yoshizawa  (1996) 
found  that  version  eye  movements  and  accommodative  response  speed  were  delayed  in  the 
course  of  40  minute  sessions  with  image  lags  of  0.3  and  0.5  s,  which  was  interpreted  as 
increased  visual  fatigue.  This  is  another  indication  for  the  load  introduced  by  large  image 
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lags.  It  is  of  interest  to  know  to  which  extent  performance  and  load  are  influenced  by  a 
realistic  image  lag. 


2  METHODS 

2.1  Simulator  and  image  presentation 

The  simulator  consists  of  an  Evans  and  Sutherland  ESIG  2000®  image  generator  with  three 
channels,  using  as  a  display  system  either  a  HMD  (Virtual  10  i-glasses®)  or  a  dome  projection 
system  (Seos  HiView  S-600®).  A  simple  dynamic  model  was  used  for  the  UAV.  It  had  a 
constant  cruising  speed  of  25  km/h,  and  its  yaw  angle  speed  (-24  to  +24  °/s)  and  vertical 
speed  (-1  to  +1  m/s)  were  controlled  by  a  joystick  (Logitech  Wingman®).  The  joystick’s 
signals  were  fed  to  a  first  order  low-pass  filter  with  a  time  constant  of  2  s,  which  served  as  the 
dynamic  model  of  the  UAV.  The  subject  was  seated  in  the  centre  of  a  spherical  screen,  on 
which  the  scene  could  be  presented.  He  wore  a  helmet  on  which  the  sensor  of  a  Polhelmus 
Fastrack®  magnetic  head  tracking  system  was  mounted.  Yaw  and  pitch  of  the  head  motion 
were  fed  into  the  image  generator. 

We  tested  the  effects  of  eight  different  viewing  conditions  on  flying  performance,  head  motion 
and  subjective  task  difficulty.  Among  these  conditions  were  two  with  a  real  HMD,  all  other 
conditions  employed  a  simulated  HMD.  In  one  real  HMD  condition  the  outside  world  scene 
was  presented  stereoscopically,  and  in  the  other  the  image  was  monoscopic,  but  presented 
dichoptically  to  both  eyes  (diagonal  field  28°).  The  additional  viewing  conditions  were 
designed  as  HMD  simulations,  set  up  to  manipulate  optical  presentation  of  the  head  slaved 
area  of  interest,  field  size  and  image  lag.  In  one  of  those  simulations,  the  HMD  was  replaced 
by  tubes  mounted  on  the  head,  in  which  an  aperture  limited  the  instantaneous  field  to  about 
the  same  size  as  the  HMD  (see  Figure  1).  The  subjects  could  scan  the  outside  world  scene 
projected  on  a  screen,  with  dimensions  151°  H  x  45°  V.  In  three  conditions  the  subject 
looked  had  a  direct  view  on  the  dome  projection.  In  these  cases,  the  instantaneous  field  of 
view  was  restricted  by  a  window  which  slaved  the  subject’s  head  motion.  Diagonal  field  sizes 
were  17°,  34°,  and  57°,  respectively.  The  (estimated)  delay  of  the  window  motion  compared 
to  the  head  motion  (70  ms)  was  determined  by  the  head  tracker’s  update  rate  and  the  image 
generator’s  image  delay.  The  conditions  with  field  size  17°  and  57°  were  also  presented  with 
an  additional  lag  on  the  image  content  of  50  ms. 
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Fig.  1  Examples  of  viewing  conditions,  a:  The  virtual  10  i-glasses  HMD;  b:  HMD 
simulation  by  means  of  FOV  restricting  spectacles;  c:  HMD  simulation  by  means  of  a 
head-tracked  window. 
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2.2  Database 

The  simulation  database  contained  32  different  routes.  They  consisted  of  successive  curved 
segments,  with  right  and  left  curves,  marked  with  trees  of  height  8.5  m,  maximal  diameter  2 
m,  and  a  inter-tree  distance  of  12  m.  The  trees  were  alternately  grouped  in  red  and  green 
rows,  red  trees  on  the  right  side  of  the  route,  green  trees  on  the  left  side.  The  terrain  was 
textured,  and  sloping.  The  route  was  in  the  middle  of  a  canyon,  limited  on  both  sides  by  a 
dike,  at  20  m  from  the  centre  of  the  course. 

The  shape  of  the  path,  including  the  slopes,  was  constructed  by  means  of  a  path  generator, 
consisting  of  the  UAV  dynamic  model  steered  by  time-varying  signals  corresponding  to  yaw 
angle  speed  and  vertical  speed.  The  signals  had  a  block  shape  which  had  passed  a  integrating 
filter  with  a  time  constant  of  2  s.  The  heights  of  the  signal  blocks  were  drawn  randomly  from 
evenly  distributed  populations,  with  maxima  equal  to  80%  of  maximum  UAV’s  yaw  angle 
speed  and  vertical  speed,  respectively.  This  procedure  guaranteed  that  it  was  physically 
possible  to  fly  the  route,  whilst  it  was  necessary  to  make  head  movements  to  determine  the 
path  curvature,  even  with  the  largest  instantaneous  field  size.  The  signal  block  lengths  were 
drawn  randomly  from  an  evenly  distributed  population,  with  the  constraint  that  the  path  s 
direction  should  not  cross  itself.  Each  route  started  with  a  straight  horizontal  part  of  75  m. 
The  length  of  each  route  was  1.7  km. 


2.3  Subjects,  task,  and  training 

Eight  paid  subjects,  male  university  students  of  age  18-25,  participated.  They  had  normal 
visual  acuity,  stereo  acuity  and  colour  vision.  Subjects  received  a  general  instruction  on  aim 
and  design  of  the  experiment,  followed  by  three  training  sessions.  The  subjects’  task  was  to 
follow  routes  marked  with  trees,  trying  to  keep  a  lateral  distance  of  3  m  to  the  trees,  and  a 
height  of  3  m  above  the  ground.  The  red  trees  were  to  be  kept  on  their  right  side,  the  green 
trees  on  their  left  side.  The  flying  time  of  each  route  was  about  4  minutes. 

In  the  training  session  subjects  followed  three  different  consecutive  routes  while  they  received 
both  sound  feedback  and  verbal  feedback  on  their  performance.  The  continuous  sound 
feedback  consisted  of  tones  provided  by  speakers  to  the  left  and  the  right  of  the  subject, 
indicating  horizontal  and  vertical  deviation  from  the  target  path,  in  steps  of  0.5  m.  Apart  from 
the  sounds,  subjects  received  verbal  feedback  on  their  performance  and  their  looking  and 
steering  strategy. 


2.4  Dependent  variables 

The  dependent  variables  consisted  of  flying  performance  measures,  head  motion  attributes, 
and  subjective  difficulty  scores. 


9 


Flying  performance.  The  following  measures  were  derived  from  comparison  of  the  UAV’s 
position  (sampled  with  10  Hz)  with  the  target  route  (horizontal  and  vertical): 

DISTH,  DISTV  the  lateral  orthogonal  error  and  the  height  error,  relative  to  the 


target  route  (m) 

mdisth,  mdistv  the  mean  over  one  run  of  DISTH  and  DISTV  (m) 

SDH,  sdv  the  standard  deviation  over  one  run  of  DISTH  and  DISTV  (m) 

SDSPEEDH,  SDSPEEDV  the  standard  deviation  over  one  run  of  lateral  and  vertical  error 

speed  (m/s). 


Head  motion  attributes.  From  the  head  yaw  and  pitch  (sampled  with  10  Hz)  the  following 


measures  were  derived: 
MYAW,  MPITCH 
SDYAW,  SDPITCH 
STDYSP,  SDTPSP 

TOTKOPMO,  STDKOPMO 


the  means  over  one  run  of  yaw  and  pitch  (°) 

the  standard  deviation  over  one  run  of  yaw  and  pitch  (°) 

the  standard  deviation  over  one  run  of  yaw  speed  and  pitch  speed 

(0/s) 

mean,  resp.  standard  deviation  over  one  run  of  the  total  speed 
vector  magnitude  (°/s) 


Subjective  difficulty  scores.  After  each  viewing  condition,  the  subject  was  asked  to  rate  the 
subjective  difficulty  on  a  scale  ranging  from  1  -  “(almost)  no  problem”  to  5  -  “(almost) 
unworkable”. 


2.5  Procedure 

Each  day  two  subjects  participated.  They  successively  received,  after  the  general  training 
session  (§  2.3),  all  eight  viewing  conditions,  and  performed  four  different  runs  (one  route  per 
run)  per  viewing  condition,  of  which  the  first  run  was  a  training  run  with  feedback  tones 
(§  2.3)  and  verbal  feedback.  Each  subject  performed  a  total  of  32  runs,  in  which  all  32 
different  routes  were  presented.  When  one  subject  was  flying  a  set  of  four  runs,  the  other 
subject  rested.  The  order  in  which  viewing  conditions  were  presented,  as  well  as  the  order  of 
routes  was  balanced  across  subjects;  the  32  different  routes  were  equally  distributed  over  all 
eight  viewing  conditions  (according  to  a  Greek-Latin  square  design). 


2.6  Statistical  design 

Analyses  of  variance  were  run  with  the  package  STATISTICA  5.0®  ANOVA/MANOVA, 
with  the  following  sub-designs  from  the  eight  viewing  conditions  (§  2.1): 
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fov  (windows  17°  -  34°  -  57°)  x  replica(3)  X  subject(8) 

DISPLAY  TYPE  (window  34°  -  tubes  -  HMD  mono)  x  replica(3)  x  subject(8) 

STEREO  (HMD  mono  -  HMD  stereo)  x  replica(3)  x  subject(8) 

lag  (no  lag  -  lag)  x  (windows  17°  -  57°)  x  replica(3)  x  subject(8) 


The  significance  of  main  effects  and  interactions  was  tested  against  the  factor  subject.  Post- 
hoc  Tukey  tests  were  performed  for  significant  effects  involving  more  than  two  levels. 


3  RESULTS 
3.1  Steering  bias 

One  may  question  whether  the  feedback  during  the  training  runs  suffices  to  learn  to  stay  on  a 
more  or  less  invisible  track.  The  results  show  that  the  average  deviation  from  the  perfect 
course  was  rather  small:  0.11  m  horizontally  and  0.42  m  vertically.  There  was  no  significant 
effect  of  viewing  condition  on  this  bias. 


3.2  Effects  of  the  factors  FOV  and  of  Display  Type 

Figures  2  a,b  show  results  for  some  of  the  performance  indicators  for  various  FOV  sizes  and 
display  types.  Figures  2  c-e  show  some  of  the  head  motion  data  and  Figure  2e  shows  the 
subjective  difficulty  ratings  made  by  the  subjects.  All  data  are  for  the  monocular  viewing  and 
no-lag  conditions. 

Performance 

Generally,  the  data  showed  a  significant  increase  in  performance  with  increasing  FOV  (see 
Table  I  and  Figures  2  a,b).  The  subjective  difficulty  ratings  agreed  very  well  with  the 
objective  performance  indicators. 


Table  I  Statistics  for  the  effects  of  the  factor  FOV  on  steering  performance. 


Performance  indicator 

F(2,14) 

p-level 

standard  deviation  of  the  lateral  error  (SDH) 

29.2  ’ 

**** 

standard  deviation  of  the  lateral  error  speed  (SDSPEEDH) 

35.2 

i 

standard  deviation  of  the  vertical  error  (SDV) 

15.7 

standard  deviation  of  the  vertical  error  speed  (SDSPEEDV) 

11.3 

** 

****  d<  .0001;  ***  ps  .001;  **  ps.01;  *  p<  .05;  nsp>.05 

s.d.  total  head  motion  (deg/s)  s.d.  head  yaw  speed(deg/s)  s.d.  lateral  error  (m) 


30  40 

FOV  (degrees) 


-o—  window 
□  tubes 
-  HMD 


FOV  (deg) 


30  4C 
FOV  (deg) 


30  40 

FOV  (deg) 


-o-  window 
n  tubes 
»  HMD 


window 
□  tubes 
»  HMD 


50  60 


FOV  (deg) 


FOV  (deg) 


Fig.  2  A  representative  overview  of  the  results  for  monocular  viewing  and  without  additional 
lag.  Performance  indicators  (a,b),  head  movement  data  (c-e)  and  subjective  difficulty  ranking 
(f)  as  a  function  of  FOV  and  display  type. 
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Post-hoc  Tukey  tests  showed  the  results  for  all  FOV  values  to  differ  significantly  from  each 
other  except  in  two  cases,  SDV  for  field  sizes  17  and  34"  and  SDSPEEDV  for  field  sizes  34 

and  57°. 

The  data  also  showed  a  remarkable  large  and  significant  (see  Table  II)  difference  between  the 
performance  with  the  simulated  HMD  (using  the  window  or  the  tube  method)  and  the  real 
HMD,  with  the  simulated  HMDs  scoring  better  than  the  real  HMD. 


Table  II  Statistics  for  the  effects  of  the  factor  display  type  on  steering  performance. 


Performance  indicator  - — - 

F(2,14) 

p-level 

standard  deviation  of  the  lateral  error 

10.4 

** 

standard  deviation  of  the  lateral  error  speed 

18.5 

standard  deviation  of  the  vertical  error 

22.5 

**** 

standard  deviation  of  the  vertical  error  speed 

10.3 

** 

****  p<  0001-  ***  p<. 001:  **  p<. 01;  *  p<.05;  ns  p>  .05 

Post-hoc  Tukey  tests  showed  the  real  HMD  to  differ  from  both  simulated  HMDs  for  all 
performance  indicators.  The  “tube  HMD”  and  the  “window  HMD”  did  not  differ  signifi¬ 
cantly  for  any  indicator. 

Head  movements 

The  FOV  had  a  significant  effect  on  most  head  movement  descriptors  (see  Table  III  and 
Figures  2  c-e).  Clearly,  the  subjects  made  less  and  slower  movements  when  the  FOV  is 

large. 


Table  III  Statistics  for  the  effects  of  the  factor  FOV  on  head  movements. 


Head  movement  descriptor  - 

F(2,14) 

p-level 

standard  deviation  of  the  yaw 

22.1 

**** 

standard  deviation  of  the  yaw  speed 

40.0 

**** 

standard  deviation  of  the  pitch 

0.22 

ns 

standard  deviation  of  the  pitch  speed 

8.99 

** 

mean  total  head  speed 

39.7 

standard  deviation  of  total  head  speed 

32.2 

— 

****  pc  0001-  ***  pc. 001;  **  pc. 01;  *  p<  .05;  ns  p>  .05 
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Generally,  the  HMD-type  did  not  have  a  significant  influence  on  head  movements.  Table  IV 
presents  the  results  of  ANOVAs  which  show  that  this  factor  has  a  significant  effect  on  the 
standard  deviation  of  the  vertical  head  velocity  only.  Post-hoc  Tukey  tests  revealed  the 
differences  between  the  real  HMD  and  the  simulated  HMDs  for  this  descriptor  to  be 
significant.  The  simulated  HMDs  did  not  differ  significantly. 


Table  IV  Statistics  for  the  effects  of  the  factor  Display  Type  on  head  movements. 


Head  movement  descriptor  - - 

F(2,14) 

p-level 

standard  deviation  of  the  yaw 

0.24 

ns 

standard  deviation  of  the  yaw  speed 

0.86 

ns 

standard  deviation  of  the  pitch 

0.93 

ns 

standard  deviation  of  the  pitch  speed 

14.3 

*** 

mean  total  head  speed 

1.43 

ns 

standard  deviation  of  total  head  speed 

0.71 

ns 

****  p<  0001-  ***  p<. 001;  **  p<. 01;  *  p<.05;  ns  p>  .05 

Subjective  difficulty 

The  subjective  difficulty  rankings  supported  the  results  on  performance  (compare  figure  1  a,b 
with  f),  in  the  sense  that  low  performance  was  coupled  with  responses  of  high  subjective 

difficulty. 


3.3  Lag 

Performance 


The  two  extreme  FOV  values  (17  and  57°)  were  used  with  and  without  an  additional  lag  of 
50  ms.  Figure  3  shows  the  resulting  performance  for  the  standard  deviation  of  the  lateral 
error  and  the  standard  deviation  of  the  lateral  error  speed.  In  both  graphs  the  influence  is 
clear.  An  ANOVA  shows  that  the  effect  of  lag  was  statistically  significant  for  three  of  the  four 
performance  indicators  (see  Table  V).  Lag  did  not  result  in  significant  subjective  difficulty 
rating  differences. 
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FOV  (deg) 


Fig.  3  Influence  of  lag  on  steering  performance. 


Table  V  Statistics  for  the  effects  of  the  factor  Lag  on  steering  performance. 


Performance  indicator  - 

F(l,7) 

p-level 

standard  deviation  of  the  lateral  error  (SDH) 

6.4 

* 

standard  deviation  of  the  lateral  error  speed  (SDSPEEDH) 

37.4 

*** 

standard  deviation  of  the  vertical  error  (SDV) 

25.4 

** 

standard  deviation  of  the  vertical  error  speed  (SDSPEEDV) 

2.8 

ns 

****  p<  0001-  ***  p< .001;  **  p<.01;  *  p< .05;  ns  p>.05 

Head  movements 

The  lag  did  not  influence  head  movements  as  indicated  by  most  of  the  head  movement 
indicators,  except  for  the  s.d.  of  the  pitch  and  the  s.d.  of  the  pitch  speed  (F(l,7)  =  12.8, 
p=  .009  and  F(l,7)=6.1,  p  =  .04,  respectively).  In  the  case  of  the  first  indicator  a  significant 
interaction  between  FOV  and  lag  was  present  (F(l,7)=6.2,  p  =  .04);  a  post-hoc  Tukey  test 
indicated  that  only  for  a  FOV  of  57°  there  is  a  significant  effect  (p  =  .0005)  of  lag.  In  this  case 
the  standard  deviation  of  the  pitch  was  lower  (2.1°)  when  lag  is  present  than  when  no  lag  was 
added  (2.8°).  The  s.d.  of  the  pitch  speed  decreased  from  2.7  °/s  to  2.0  °/s  when  a  lag  was 
added.  There  was  no  significant  interaction  of  FOV  and  lag  in  this  case. 

Subjective  difficulty 

Lag  did  not  significantly  influence  subjective  difficulty  ratings. 
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3.4  Stereoscopic  vs.  monocular  HMD  presentation 

None  of  the  performance  indicators  indicated  a  difference  between  monocular  and  stereo¬ 
scopic  HMD  presentation.  As  far  as  head  movements  are  concerned,  only  in  one  case  did  the 
data  show  a  significant  effect:  The  average  pitch  in  the  stereo  condition  was  1.5  higher  than 
in  the  monocular  viewing  condition  (F(l,7)=20.5,  p  =  .003).  This  was  probably  a  spurious 
result.  The  subjective  difficulty  ranking  did  not  indicate  a  preference  for  either  monocular  or 
stereoscopic  presentation. 


4  CONCLUSIONS  AND  DISCUSSION 

We  have  measured  the  influence  of  Field  Of  View,  image  lag,  monocular  and  stereoscopic 
presentation,  and  display  type  on  several  performance  indicators,  head  movement  descriptors 
and  subjective  difficulty  ratings.  Of  these  factors,  FOV  has  a  profound  effect  on  the  measure¬ 
ments,  with  less  steering  errors  and  less  head  movements  with  higher  FOVs.  The  same  clear 
results  are  found  for  the  display  types,  with  the  simulated  HMDs  scoring  noticeably  better 
than  the  real  HMD.  The  effect  of  lag  is  clear  in  the  performance  data,  but  not  in  the  head 
movement  data.  There  is  no  apparent  effect  of  monocular  or  stereoscopic  presentation. 

Field  Of  View 


The  effects  of  the  FOV  on  head  movements  are  easily  explained.  For  a  particular  FOV  the 
movement  behaviour  is  dictated  by  the  course  ahead  (see  Figure  4).  The  smaller  the  FOV  the 
less  the  amount  of  information  about  the  approaching  course  features  can  be  acquired 
instantaneously  and  therefore  more  and  faster  head  movements  are  necessary  to  collect  the 
required  information  in  time. 

The  effects  of  the  FOV  on  the  steering  accuracy  are  less  easily  explained.  In  principle,  all  the 
information  present  in  the  case  of  a  large  FOV  can  be  gathered  by  increasing  the  amount  of 
head  movements  if  the  FOV  is  small.  Four  possible  explanations  for  a  lower  performance  at 
small  FOVs  are: 

1  Initiating  and  controlling  the  increased  amount  of  head  movements  increases  the  workload 
which  interferes  with  the  steering  task.  Normally,  head  movements  are  both  controlled  by 
attention  (foveal  selection)  and  by  events  in  the  outer  periphery  of  the  retina.  With  HMDs 
the  FOV  is  too  small  to  cover  the  periphery  and  therefore  head  movements  need  to  be 
initiated  more  consciously.  Furthermore,  some  subjects  reported  interference  of  head 
motion  with  their  perception  of  vehicle  motion. 


Heading  (deg) 
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Fig.  4  A  closer  analysis  of  the  head  movements  of  a  subject  during  a  particular  run  using  a 
simulated  HMD  with  FOV  of  17  degrees.  Please  note  that  the  plot  only  indicates  the  head 
direction  and  not  the  direction  of  the  eyes,  a:  A  plot  of  heading  and  pitch  directions  showing  a 
tendency  to  fixate  in  the  forward  direction  and  some  rare  extreme  movements  in  either  pitch  and 
heading  directions,  b:  A  heading  vs.  time  plot  reveals  that  the  heading  is  close  to  zero  for  most 
of  the  time  except  for  some  rapid  movements  at  the  end  of  the  run.  c:  A  plot  of  the  actual  flown 
track  (starting  point  of  the  arrows),  the  ideal  track  and  the  general  viewing  direction  (indicated 
by  the  arrows).  This  plot  explains  the  sharp  movements  at  the  end  of  the  heading-time  plot.  High 
heading  values  are  necessary  when  the  route  is  changing  sharply . 


2  The  increased  amount  of  head  movements  does  not  fully  compensate  for  the  loss  of 
information  content  due  to  a  smaller  FOV.  This  hypothesis  is  supported  by  the  data:  the 
horizontal  viewing  angle  range  (here  loosely  defined  as  the  range  in  which  95  %  of  the 
head  movements  can  be  found  plus  the  instantaneous  FOV)  is  59.7°,  64.6°  and  76.4°  for 
the  small,  mid-size  and  large  FOV,  respectively.  Even  if  we  examine  the  full  range  of 
head  movements  (extreme  values)  is  the  horizontal  viewing  range  about  10°  larger  for  the 
large  FOV  than  for  the  small  FOV.  This  is  much  clearer  in  the  pitch  data.  The  s.d.  of  the 
pitch  showed  no  effect  of  FOV,  which  means  that  the  subjects  did  not  compensate  for 
decreased  vertical  FOV.  Compensation  of  decreasing  vertical  FOV  was  probably  less 
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necessary  since  the  largest  changes  in  the  course  were  in  horizontal  direction  and  not  in  the 
vertical  direction. 

3  The  increased  amplitude  and  velocity  of  the  head  movements  decreases  the  accuracy  with 
which  visual  data  is  acquired  and  used.  It  is  known  that  head  movements  may  lead  to 
considerable  retinal  slip.  However,  it  seems  that  this  retinal  slip  does  not  diminish  visual 
acuity  considerably  (Sperling,  1990;  Steinman  &  Collewijn,  1980). 

4  Increased  head  motions  may  lead  to  (subconscious)  symptoms  of  simulator  sickness.  One 
of  the  subjects  got  sick  during  the  experiments  while  using  a  small  FOV.  He  was  replaced 
by  another  subject  who,  as  all  the  others,  did  not  show  similar  symptoms.  Nevertheless,  a 
slight  discomfort  may  play  a  role. 

Image  lag 

Although  our  head  slaving  lag  was  rather  small  at  50  ms,  it  resulted  in  noticeable  performance 
loss  (about  20%  higher  error  scores)  and  a  small  reduction  in  head  movements  (25  %  less 
pitch  standard  deviation  with  lag,  but  then  only  in  the  57°  case  and  no  significant  influence  on 
head  yaw  movements).  Grunwald  et  al.  (1991),  using  a  lag  of  500  ms,  found  a  small  influence 
on  performance  (4%  higher  error  scores)  but  a  large  impact  on  head  movements  (a  53% 
lower  head  yaw  rate).  The  differences  in  head  movements  may  be  explained  by  the  lower  lag 
value  we  used.  The  higher  steering  error  rates  we  obtained  may  be  due  to  the  higher 
importance  for  steering  accuracy  of  the  head  movements  in  our  set  up. 

Monocular  vs.  stereoscopic  presentation 

Stereoscopic  presentation  did  not  improve  the  steering  performance.  This  can  not  be 
explained  by  the  fact  that  the  subjects  made  head  movements.  Although  Steinman  and 
Collewijn  (1980)  show  that  head  rotations  lead  to  retinal  slip  and  that  vergence  is  not  kept 
stable  during  the  movements,  their  results  and  those  of  Patterson  and  Fox  (1984)  and  others 
(see  e.g.  Regan,  Frisby,  Poggio,  Schor  &  Tyler,  1990)  indicate  that  stereo-acuity  is  not 
impaired  by  head  movements. 

Calculations  of  optimal  just  noticeable  depth  differences  (jnd)  for  the  Virtual  IO  HMD,  based 
on  its  addressable  resolution,  show  that  at  a  distance  of  3  m  a  depth  jnd  of  about  9  cm  could 
be  achieved.  The  s.d.  of  the  lateral  error  is  2-4  m,  so  the  resolution  may  not  seem  to  restrict 
the  utility  of  the  stereoscopic  presentation.  However,  in  course  planning  pilots  and  drivers 
look  forward  in  time.  Values  of  about  2-5  s  are  quite  common,  which  in  our  case  means 
looking  ahead  approximately  15-40  m.  The  jnd  at  these  distance  is  3-24  m,  so  it  follows  that 
stereo  given  the  resolution  of  the  Virtual  IO  HMD  is  probably  of  not  much  use.  Furthermore, 
the  monocular  cues  in  the  scene  were  quite  strong;  all  trees  were  of  the  same  size,  and 
therefor  their  size  in  projection  corresponded  directly  with  distance.  The  effect  of  a  stereo  cue 
may  have  been  drowned  in  the  effects  of  this  monocular  cue. 
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Display  type 

A  part  of  the  lower  performance  of  the  real  HMD  as  compared  to  the  performance  with 
simulated  HMDs  may  be  explained  by  the  almost  unnoticeable  motion  smear  present  in  the 
HMD.  This  smear  is  due  to  the  inertia  of  the  LCD  display.  In  any  case,  the  results  can  not  be 
explained  by  resolution  differences.  The  addressable  vertical  resolution  of  the  real  HMD  is 
higher  by  a  factor  of  almost  2  compared  to  the  simulated  HMDs.  Moreover,  Van  Erp  (1996) 
showed  for  manoeuvring  tasks  that  drivers  tolerate  a  reduction  of  the  resolution  by  a  factor  of 
2  without  loss  of  performance.  To  explain  the  results  in  terms  of  resolution  would  therefore 
imply  that  the  VGA  to  NTSC  conversion  needed  to  drive  the  Virtual  10  HMD  lowers  the 
resolution  by  at  least  a  factor  of  4,  which  would  have  been  clearly  noticeable.  Since  this  was 
not  the  case,  and  because  all  objects  in  the  database  were  rather  large  and  visible  distances 
short,  we  may  exclude  resolution  as  an  explanation. 

With  the  simulated  HMDs,  vergence  and  accommodation  of  the  eyes  match.  Both  are  at  a 
distance  of  about  3  m.  For  the  Virtual  10  HMD  (Kooi,  1996),  convergence  is  at  2.5  m  and 
accommodation  at  4  m.  Accommodation  is  a  weak  cue  which  does  not  have  much  influence  at 
distances  over  1-2  m.  Effects  of  convergence  have  a  slightly  larger  range,  but  the  effects  of 
the  differences  between  convergence  of  real  and  simulated  HMDs  are  too  small  to  account  for 
the  difference  in  performance. 

We  conjecture  that  the  simulated  HMDs  work  better  because  they  give  the  subjects  the 
possibility  to  orient  themselves  in  space:  the  projection  screen  is  dimly  visible  and  its  centre 
corresponds  with  the  heading  direction  of  the  UAV.  When  using  the  HMD,  no  such  visual 
orientation  marks  are  present.  Since  the  optic  flow  is  a  combination  of  head  motion  and 
vehicle  motion  and  the  head  motion  may  not  be  accurately  represented  internally,  estimation 
of  the  heading  direction  will  be  more  difficult  for  a  real  HMD  than  for  a  simulated  HMD.  It 
suggests  that  users  of  immersive  HMDs  should  be  provided  with  reference  marks.  This 
hypothesis  will  be  tested  in  a  future  experiment. 
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